The present invention relates to body-worn mobile hearing devices, such as head-worn hearing aids and personal sound amplifier products (“PSAPs”) which are powered by one or more batteries. In particular, the present invention pertains to such hearing devices which have the capacity to wirelessly stream real-time audio content.
Today's hearing devices are increasingly technologically advanced to improve the overall experience of the user. Such hearing devices are commonly powered by a rechargeable or replaceable battery, powering a digital signal processor and a small speaker (referred to in the hearing aid industry as a “receiver”) with a sound outlet typically located in the ear canal, and commonly also including a microphone to receive airbourne audio pressure waves. Changing the battery in such devices can range from being a nuisance to being extremely difficult, depending upon the manual dexterity of the user (who is frequently elderly). Frequent battery changes also increases operating expense. Accordingly, such hearing devices should be designed to use as little electrical power as possible and thereby conserve battery life for as long as possible. Such hearing devices are frequently also quite small, thereby making the device more inconspicuous and oftentimes more comfortable for head-worn or ear-carried wearing.
One common technological advancement which is desired to be included in such mobile hearing devices is an integrated wireless system. In one aspect, an on-board integrated wireless system can enable commands and programming data to be wirelessly communicated to the hearing device, such as for changing various settings or programs of the digital signal processor. The recent standardization of a Bluetooth Low Energy (“BLE”) protocol has created new opportunities for sending various types of data to a hearing device while conserving battery power. BLE is designed, standardized and marketed by the Bluetooth Special Interest Group (“SIG”), and is intended to provide considerably reduced power consumption and cost while maintaining a similar communication range (generally less than 100 m) as classic Bluetooth. BLE operates in the same spectrum range (approximately 2.4 GHz) as classic Bluetooth technology, but generally has 40 2-MHz channels instead of 79 1-MHz channels. Within a channel, data is transmitted using Gaussian frequency shift keying modulation, using an adaptive frequency hopping scheme to counteract narrowband interference problems. Each BLE packet must be from 80 to 376 bits in length.
Typical BLE applications are provided with a BLE protocol stack that can be split between defined host and controller roles. The host of the BLE protocol stack (typically running on an application processor) generally includes the Generic Access Profile (GAP), the Generic Attribute Profile (GATT), the Attribute Protocol (ATT), the Security Manager (SM) and the Logical Link Control and Adaption Protocol (L2CAP). The controller of the BLE protocol stack (typically implemented as a small System-on-Chip (SOC) with an integrated radio) includes a Link Layer (LL) and a Physical Layer (PHY). The PHY in BLE operates at a data rate of 1 Mbps. Communication between the Host and the Controller is standardized via the Host Controller Interface (HCI). The Link Layer in the BLE stack of the hearing device is responsible for managing the physical transceiver radio components (PHY), i.e, for managing all of the timing and synchronization, frequency hopping algorithms, state machines, send and receive Channel Protocol Data Units (Advertising and Data channels), assembly and transfer of packets to and from the PHY and L2CAP and numerous other functions. The activity between the Link Layer and corresponding channel activity on the PHY are quite advanced, and depend on the current state and connection parameters—all of which the Link Layer knows about and manages by being told these settings from L2CAP instructions. All connection activity, whether it is an advertisement or a connection event, happens in synchronous time intervals, that is, repeatable time frames. This is a key aspect of BLE and is what allows the battery life saving that has made BLE so popular. If we look within a connection event—again either for advertising packets or connection data packets—there can be a number of exchanges within the event. The event always starts by a master sending a packet to the slave, starting at time 0, called the anchor point. The slave, in this case the mobile hearing device, can and will respond as needed for that connection event. The connection event can be ended by either the slave or the master, and when it does end, the PHY turns off until the next anchor point. The key to BLE's power savings is the duty cycle, or time comparison between the length of the connection event and the connection interval. In some real world applications, such as sending settings of the digital signal processor of a hearing aid or PSAP, the connection event is often a 300 μS maximum activity duration within a 500 mS connection interval, which corresponds to 0.06% duty cycle.
Several of the constraints of BLE are important toward determining theoretical and realistic maximum data throughput rate. In particular, the time between two consecutive connection events between a master and slave is defined in the BLE 4.0 specification to be between 7.5 ms and 4.0 s in 1.25 ms increments. In two popular market implementations, Android operating system smartphones have a minimum time between two consecutive BLE connection events of 7.5 ms, whereas APPLE iOS smartphones have a minimum time between two consecutive BLE connection events of 18.5 ms. The maximum L2CAP payload size is defined in the BLE 4.0 specification to be 23 bytes. Above the L2CAP layer, the Attribute Protocol imposes a 3 byte header, leaving 20 bytes for maximum application data per packet. Including Inter Frame Space time for silicon cool down and for acknowledgements for each PDU as defined in BLE 4.0, this results in a maximum theoretic application data throughput rate of about 234 kbps. Discounted for real world conditions such as Bit Error Rates (BLE behavior limits unnecessary waste of energy while bit errors are being found in one data channel, but degrades the effective throughput of BLE in the presence of bit errors), the number of application layer messages a device can send per connection event (due to memory limitations), the actual number of notifications per connection event, as well as processing delays, the maximum BLE throughput that can be reliably achieved in a real scenario is about 34 kbps for unacknowledged data transmission and about 8.5 kbps for acknowledged data transmission.
Separately, the constraints of sampling, transmitting and receiving audio information are generally well know and well established. The design inputs and requirements for streaming audio for hearing aid applications, such as when viewing an audio-visual program including speech, include significant latency and audio quality limitations.
Streaming of audio for hearing aid applications, and in most other similar consumer electronics, has a challenging requirement for latency. Latency is defined as the time it takes to convert sound waves from the streaming device (source), transmit the digitized information wirelessly to the hearing aid (sink), and convert back into an understandable recreation of that original audio using the hearing aid receiver (speaker). Latency becomes increasingly important for applications that involve sound and visual synchronization. For example, when a user watches a TV, the user can see the content visually, and needs to receive the corresponding audio within a “reasonable amount of time”, before the user experience is compromised by a noticeable audio delay. The hearing aid industry has researched this topic in great detail, and has concluded that the maximum latency before a user begins to notice the delay of audio from corresponding visual cues depends greatly on age but ranges from 5 to 40 ms. The Bluetooth SIG Hearing Aid Working Group (HAWG), which includes six leading hearing aid manufacturers, has agreed on a maximum wireless system delay of 20 ms for future Bluetooth standards. The wireless system delay can include delays associated with encoding the audio signal, packetizing the audio data, transmitting the audio data, receiving the audio data, reassembling the packets, and decoding the audio data. Additional delays can be incurred in the hearing device due to processing the audio data (in the digital signal processor fit for the particular user) and converting the audio data into sound. Because of the challenging latency requirements of audio streaming, the connection interval of audio packets must be less than 40 ms to achieve the latency requirement, or 20 ms to achieve the requirement of typical hearing aid applications. Further, it is likely that packets and events will—at times—fail to be received by the hearing device, and thus some redundancy is required in multiple packets and events to reduce the impact of lost packets. The 20 mS interval can be broken down into quarters to enable this redundancy, resulting in synchronous events separated by 5 ms. Longer event intervals such as 10 ms may be used (or anything in between), but longer event intervals make redundancy more difficult to achieve.
In the hearing aid industry and in general consumer electronics like Bluetooth headsets, audio quality has been heavily researched to determine the quality requirements for the applications and markets of interest. For hearing aids, there are two quality definitions considered; standard quality (SQ) and high quality (HQ). In typical audio and hearing aid applications, a 16 bit analog to digital converter at a sampling frequency of 16 kHz (24 kHz for HQ) can achieve streaming audio frequencies up to 8 kHz, an acceptable maximum frequency and acceptable fidelity for the human ear including for speech. Combining these two considerations, the minimum uncompressed digital audio data is 16 bit×16 khz (SQ)=256 kbps, or 16 bit×24 khz (HQ)=348 kbps.
The hearing industry has also researched audio compression algorithms to reduce the wireless throughput requirements, while still maintaining the ability to acceptably recreate the original audio signal. Industry standard codecs like G722 can achieve 4:1 compression ratios, though compression ratios up to 6:1 or even 8:1 may be achievable in the future. Combining the industry standard codec with and audio requirements, the required encoded data reliable transmission rate is 256 kbps/(4:1)=64 kbps (SQ), or 348 kbps/(4:1)=96 kbps (HQ). The above value of 64 kbps should be considered the minimum amount of data that the wireless system presently needs to transfer to the hearing device (sink) in a streaming mode to achieve “real-time streaming audio”. Note that this value can and will increase for various other considerations, such as for including header information, and for stereo streaming mode and HQ streaming, but could decrease with better (future, i.e., unknown to Applicant at this time) codecs.
Given these latency and quality data transfer limitations, BLE is generally viewed as being inadequate to transfer audio information, particularly real-time streaming of audio information. Accordingly, most mobile hearing devices, if they support wireless audio streaming, transmit the audio information to the mobile hearing device via a proprietary protocol or link. If a mobile hearing device supports both BLE and proprietary transmissions, there is generally separate hardware for both, increasing the cost, complexity and circuit footprint of the device.
Examples which seek to merge BLE with audio streaming transmissions include U.S. Pat. No. 8,849,202, U.S. Pat. Pub. Nos. 2013/0102251, 2014/0348327, 2015/0010179, 2015/0201446 and 2015/0334488, and WIPO Pat. Pub. No. WO2014/076527, each incorporated by reference, but none of such examples consider the present invention. While these examples have various benefits and (particularly feasibility) detriments, better solutions are needed.